The Weighted Sparsity Problem: Complexity and Algorithms

Many optimization algorithms involve repeated processing of a fixed set of linear constraints. If we pre-process the constraint matrix A to make it sparser, algebraic operations should become faster. In many applications there is a priori information about the likelihood that each column will appear in a basis, which can be expressed as weights on the columns. This leads to considering the Weighted Sparsity Problem (WSP): Find a row-equivalent constraint matrix with as small a weight of non-zeros as possible. We show that WSP is NP-hard even with a non-degeneracy assumption, and even if restricted to instances with at most three non-zeros per both row and column. WSP is shown to have a polynomial algorithm when the number of non-zeros per either row or column is limited to at most two. This contrasts with previous results that, assuming only non-degeneracy, the unweighted version of WSP does have a polynomial algorithm (this has proven to be practically useful in tests on real data). The polynomial algorithm for WSP with at most two non-zeros per row or column is based on solving one-row problems via minimum cut calculations, together with a sufficient condition for piecing these one-row solutions together into a global solution.